Sampling Equivariant Self-Attention Networks for Object Detection in Aerial Images.

Objects in aerial images show greater variations in scale and orientation than in other images, making them harder to detect using vanilla deep convolutional neural networks. Networks with sampling equivariance can adapt sampling from input feature maps to object transformation, allowing a convolutional kernel to extract effective object features under different transformations. However, methods such as deformable convolutional networks can only provide sampling equivariance under certain circumstances, as they sample by location. We propose sampling equivariant self-attention networks, which treat self-attention restricted to a local image patch as convolution sampling by masks instead of locations, and a transformation embedding module to improve the equivariant sampling further. We further propose a novel randomized normalization module to enhance network generalization and a quantitative evaluation metric to fairly evaluate the ability of sampling equivariance of different models. Experiments show that our model provides significantly better sampling equivariance than existing methods without additional supervision and can thus extract more effective image features. Our model achieves state-of-the-art results on the DOTA-v1.0, DOTA-v1.5, and HRSC2016 datasets without additional computations or parameters.

Anti-factor Xa level monitoring of low-molecular-weight heparin for prevention of venous thromboembolism in critically ill patients (AXaLPE): protocol of a randomised, open-label controlled clinical trial.

INTRODUCTION: Whether and when to monitor the amount of anti-factor Xa (aFXa) activity in critically ill patients with complex diseases to prevent venous thromboembolism (VTE) remain unclear. This study is a randomised controlled trial to investigate the effect of aFXa level monitoring on reducing VTE and to establish a new method for accurately preventing VTE in critically ill patients with low-molecular-weight heparin (LMWH). METHODS AND ANALYSIS: A randomised controlled trial is planned in two centres with a planned sample size of 858 participants. Participants will be randomly assigned to three groups receiving LMWH prophylaxis at a 1:1:1 ratio: in group A, peak aFXa levels will serve as the guide for the LMWH dose; in group B, the trough aFXa levels will serve as the guide for the LMWH dose; and in group C, participants serving as the control group will receive a fixed dose of LMWH. The peak and trough aFXa levels will be monitored after LMWH (enoxaparin, 40 mg, once daily) reaches a steady state for at least 3 days. The monitoring range for group A's aFXa peak value will be 0.3-0.5 IU/mL, between 0.1 and 0.2 IU/mL is the target range for group B's aFXa trough value. In order to reach the peak or trough aFXa levels, groups A and B will be modified in accordance with the monitoring peak and trough aFXa level. The incidence of VTE will serve as the study's primary outcome indicator. An analysis using the intention-to-treat and per-protocol criterion will serve as the main outcome measurement. ETHICS AND DISSEMINATION: The Xuanwu Hospital Ethics Committee of Capital Medical University and Peking University First Hospital Ethics Committee have approved this investigation. It will be released in all available worldwide, open-access, peer-reviewed publications. TRIAL REGISTRATION NUMBER: NCT05382481.

Skeleton-CutMix: Mixing Up Skeleton With Probabilistic Bone Exchange for Supervised Domain Adaptation.

We present Skeleton-CutMix, a simple and effective skeleton augmentation framework for supervised domain adaptation and show its advantage in skeleton-based action recognition tasks. Existing approaches usually perform domain adaptation for action recognition with elaborate loss functions that aim to achieve domain alignment. However, they fail to capture the intrinsic characteristics of skeleton representation. Benefiting from the well-defined correspondence between bones of a pair of skeletons, we instead mitigate domain shift by fabricating skeleton data in a mixed domain, which mixes up bones from the source domain and the target domain. The fabricated skeletons in the mixed domain can be used to augment training data and train a more general and robust model for action recognition. Specifically, we hallucinate new skeletons by using pairs of skeletons from the source and target domains; a new skeleton is generated by exchanging some bones from the skeleton in the source domain with corresponding bones from the skeleton in the target domain, which resembles a cut-and-mix operation. When exchanging bones from different domains, we introduce a class-specific bone sampling strategy so that bones that are more important for an action class are exchanged with higher probability when generating augmentation samples for that class. We show experimentally that the simple bone exchange strategy for augmentation is efficient and effective and that distinctive motion features are preserved while mixing both action and style across domains. We validate our method in cross-dataset and cross-age settings on NTU-60 and ETRI-Activity3D datasets with an average gain of over 3% in terms of action recognition accuracy, and demonstrate its superior performance over previous domain adaptation approaches as well as other skeleton augmentation strategies.

LC-NeRF: Local Controllable Face Generation in Neural Radiance Field.

3D face generation has achieved high visual quality and 3D consistency thanks to the development of neural radiance fields (NeRF). However, these methods model the whole face as a neural radiance field, which limits the controllability of the local regions. In other words, previous methods struggle to independently control local regions, such as the mouth, nose, and hair. To improve local controllability in NeRF-based face generation, we propose LC-NeRF, which is composed of a Local Region Generators Module (LRGM) and a Spatial-Aware Fusion Module (SAFM), allowing for geometry and texture control of local facial regions. The LRGM models different facial regions as independent neural radiance fields and the SAFM is responsible for merging multiple independent neural radiance fields into a complete representation. Finally, LC-NeRF enables the modification of the latent code associated with each individual generator, thereby allowing precise control over the corresponding local region. Qualitative and quantitative evaluations show that our method provides better local controllability than state-of-the-art 3D-aware face generation methods. A perception study reveals that our method outperforms existing state-of-the-art methods in terms of image quality, face consistency, and editing effects. Furthermore, our method exhibits favorable performance in downstream tasks, including real image editing and text-driven facial image editing.

Mesh Neural Networks Based on Dual Graph Pyramids.

Deep neural networks (DNNs) have been widely used for mesh processing in recent years. However, current DNNs can not process arbitrary meshes efficiently. On the one hand, most DNNs expect 2-manifold, watertight meshes, but many meshes, whether manually designed or automatically generated, may have gaps, non-manifold geometry, or other defects. On the other hand, the irregular structure of meshes also brings challenges to building hierarchical structures and aggregating local geometric information, which is critical to conduct DNNs. In this paper, we present DGNet, an efficient, effective and generic deep neural mesh processing network based on dual graph pyramids; it can handle arbitrary meshes. Firstly, we construct dual graph pyramids for meshes to guide feature propagation between hierarchical levels for both downsampling and upsampling. Secondly, we propose a novel convolution to aggregate local features on the proposed hierarchical graphs. By utilizing both geodesic neighbors and Euclidean neighbors, the network enables feature aggregation both within local surface patches and between isolated mesh components. Experimental results demonstrate that DGNet can be applied to both shape analysis and large-scale scene understanding. Furthermore, it achieves superior performance on various benchmarks, including ShapeNetCore, HumanBody, ScanNet and Matterport3D. Code and models will be available at https://github.com/li-xl/DGNet.

On Rotation Gains Within and Beyond Perceptual Limitations for Seated VR.

Head tracking in head-mounted displays (HMDs) enables users to explore a 360-degree virtual scene with free head movements. However, for seated use of HMDs such as users sitting on a chair or a couch, physically turning around 360-degree is not possible. Redirection techniques decouple tracked physical motion and virtual motion, allowing users to explore virtual environments with more flexibility. In seated situations with only head movements available, the difference of stimulus might cause the detection thresholds of rotation gains to differ from that of redirected walking. Therefore we present an experiment with a two-alternative forced-choice (2AFC) design to compare the thresholds for seated and standing situations. Results indicate that users are unable to discriminate rotation gains between 0.89 and 1.28, a smaller range compared to the standing condition. We further treated head amplification as an interaction technique and found that a gain of 2.5, though not a hard threshold, was near the largest gain that users consider applicable. Overall, our work aims to better understand human perception of rotation gains in seated VR and the results provide guidance for future design choices of its applications.

Real-Time Globally Consistent 3D Reconstruction With Semantic Priors.

Maintaining global consistency continues to be critical for online 3D indoor scene reconstruction. However, it is still challenging to generate satisfactory 3D reconstruction in terms of global consistency for previous approaches using purely geometric analysis, even with bundle adjustment or loop closure techniques. In this article, we propose a novel real-time 3D reconstruction approach which effectively integrates both semantic and geometric cues. The key challenge is how to map this indicative information, i.e., semantic priors, into a metric space as measurable information, thus enabling more accurate semantic fusion leveraging both the geometric and semantic cues. To this end, we introduce a semantic space with a continuous metric function measuring the distance between discrete semantic observations. Within the semantic space, we present an accurate frame-to-model semantic tracker for camera pose estimation, and semantic pose graph equipped with semantic links between submaps for globally consistent 3D scene reconstruction. With extensive evaluation on public synthetic and real-world 3D indoor scene RGB-D datasets, we show that our approach outperforms the previous approaches for 3D scene reconstruction both quantitatively and qualitatively, especially in terms of global consistency.

Adaptive Optimization Algorithm for Resetting Techniques in Obstacle-Ridden Environments.

Redirected Walking (RDW) algorithms aim to impose several types of gains on users immersed in Virtual Reality and distort their walking paths in the real world, thus enabling them to explore a larger space. Since collision with physical boundaries is inevitable, a reset strategy needs to be provided to allow users to reset when they hit the boundary. However, most reset strategies are based on simple heuristics by choosing a seemingly suitable solution, which may not perform well in practice. In this article, we propose a novel optimization-based reset algorithm adaptive to different RDW algorithms. Inspired by the approach of finite element analysis, our algorithm splits the boundary of the physical world by a set of endpoints. Each endpoint is assigned a reset vector to represent the optimized reset direction when hitting the boundary. The reset vectors on the edge will be determined by the interpolation between two neighbouring endpoints. We conduct simulation-based experiments for three RDW algorithms with commonly used reset algorithms to compare with. The results demonstrate that the proposed algorithm significantly reduces the number of resets.

Multiway Non-Rigid Point Cloud Registration via Learned Functional Map Synchronization.

We present SyNoRiM, a novel way to jointly register multiple non-rigid shapes by synchronizing the maps that relate learned functions defined on the point clouds. Even though the ability to process non-rigid shapes is critical in various applications ranging from computer animation to 3D digitization, the literature still lacks a robust and flexible framework to match and align a collection of real, noisy scans observed under occlusions. Given a set of such point clouds, our method first computes the pairwise correspondences parameterized via functional maps. We simultaneously learn potentially non-orthogonal basis functions to effectively regularize the deformations, while handling the occlusions in an elegant way. To maximally benefit from the multi-way information provided by the inferred pairwise deformation fields, we synchronize the pairwise functional maps into a cycle-consistent whole thanks to our novel and principled optimization formulation. We demonstrate via extensive experiments that our method achieves a state-of-the-art performance in registration accuracy, while being flexible and efficient as we handle both non-rigid and multi-body cases in a unified framework and avoid the costly optimization over point-wise permutations by the use of basis function maps.

Recursive-NeRF: An Efficient and Dynamically Growing NeRF.

View synthesis methods using implicit continuous shape representations learned from a set of images, such as the Neural Radiance Field (NeRF) method, have gained increasing attention due to their high quality imagery and scalability to high resolution. However, the heavy computation required by its volumetric approach prevents NeRF from being useful in practice; minutes are taken to render a single image of a few megapixels. Now, an image of a scene can be rendered in a level-of-detail manner, so we posit that a complicated region of the scene should be represented by a large neural network while a small neural network is capable of encoding a simple region, enabling a balance between efficiency and quality. Recursive-NeRF is our embodiment of this idea, providing an efficient and adaptive rendering and training approach for NeRF. The core of Recursive-NeRF learns uncertainties for query coordinates, representing the quality of the predicted color and volumetric intensity at each level. Only query coordinates with high uncertainties are forwarded to the next level to a bigger neural network with a more powerful representational capability. The final rendered image is a composition of results from neural networks of all levels. Our evaluation on public datasets and a large-scale scene dataset we collected shows that Recursive-NeRF is more efficient than NeRF while providing state-of-the-art quality. The code will be available at https://github.com/Gword/Recursive-NeRF.

Beyond Self-Attention: External Attention Using Two Linear Layers for Visual Tasks.

Attention mechanisms, especially self-attention, have played an increasingly important role in deep feature representation for visual tasks. Self-attention updates the feature at each position by computing a weighted sum of features using pair-wise affinities across all positions to capture the long-range dependency within a single sample. However, self-attention has quadratic complexity and ignores potential correlation between different samples. This article proposes a novel attention mechanism which we call external attention, based on two external, small, learnable, shared memories, which can be implemented easily by simply using two cascaded linear layers and two normalization layers; it conveniently replaces self-attention in existing popular architectures. External attention has linear complexity and implicitly considers the correlations between all data samples. We further incorporate the multi-head mechanism into external attention to provide an all-MLP architecture, external attention MLP (EAMLP), for image classification. Extensive experiments on image classification, object detection, semantic segmentation, instance segmentation, image generation, and point cloud analysis reveal that our method provides results comparable or superior to the self-attention mechanism and some of its variants, with much lower computational and memory costs.

Safety of inactivated SARS-CoV-2 vaccines in myasthenia gravis: A survey-based study.

Background: Vaccination remains the most effective measure to prevent SARS-CoV-2 infection and worse outcomes. However, many myasthenia gravis (MG) patients are hesitant to receive vaccine due to fear of worsening. Methods: MG patients were consecutively enrolled in two MG centers in North China. The "worsening" after vaccination was self-reported by MG patients, and severity was measured with a single simple question. The general characteristics and disease status immediately prior to the first dose were compared between the worsening and non-worsening groups. Independent factors associated with worsening were explored with multivariate regression analysis. Results: One hundred and seven patients were included. Eleven patients (10.3%) reported worsening after vaccination, including eight patients with mild or moderate worsening and three patients with severe worsening. Only one of them (0.9%) needed an escalation of immunosuppressive treatments. There were significant differences between the worsening and non-worsening groups in terms of Myasthenia Gravis Foundation of America classes immediately before the first dose and intervals since the last aggravation. Precipitating factors might contribute to the worsening in some patients. Logistic regression revealed that only interval since the last aggravation ≤6 months was associated with worsening after SARS-CoV-2 vaccination (P = 0.01, OR = 8.62, 95% CI: 1.93-38.46). Conclusion: SARS-CoV-2 vaccines (an overwhelming majority were inactivated vaccines) were found safe in milder Chinese MG patients who finished two doses. Worsening after vaccination was more frequently seen in patients who were presumed as potentially unstable (intervals since last aggravation ≤6 months). However, mild worsening did occur in patients who were presumed to be stable. Precipitating factors should still be sought and treated for better outcome.

Frontoparietal paired associative stimulation versus single-site stimulation for generalized anxiety disorder: a pilot rTMS study.

BACKGROUND: At present, the use of repetitive transcranial magnetic stimulation (rTMS) for generalized anxiety disorder (GAD) is limited to single-site interventions. We investigated whether dual-site frontoparietal stimulation delivered using cortical-cortical paired associative stimulation (ccPAS) had stronger clinical efficacy than single-site stimulation in patients with GAD. METHODS: We randomized 50 patients with GAD to 1 Hz rTMS (10 sessions) using 1 of the following protocols: single-site stimulation over the right dorsolateral prefrontal cortex (dlPFC; 1500 pulses per session); single-site stimulation over the right posterior parietal cortex (PPC; 1500 pulses per session); repetitive dual-site ccPAS (rds-ccPAS) over the right dlPFC and right PPC with 1500 pulses per session (rd-ccPAS-1500); or rds-ccPAS over the right dlPFC and right PPC with 750 pulses per session (rd-ccPAS-750). Both rds-ccPAS treatments used a between-site interval of 100 ms. RESULTS: Clinical scores for anxiety, depression and insomnia were reduced in all 4 groups after treatment. We found greater improvements in anxiety symptoms in the rds-ccPAS-1500 group compared to the rds-ccPAS-750 and single-site groups. We found greater improvements in depression symptoms and insomnia in the rds-PAS-1500 group compared to the single-site groups. The rds-ccPAS-1500 group also showed significant or trend-level improvements in anxiety symptoms and insomnia at 10-day and 1-month followup. More patients responded to treatment with rds-ccPAS-1500 than with single-site stimulation. The between-group differences in response rates persisted to the 3-month follow-up. Treatment using rds-ccPAS with a between-site interval of 100 ms induced a more significant improvement than the between-site interval of 50 ms we evaluated in a previous study. LIMITATIONS: These results need to be replicated in a larger sample using sham control and equal-pulse single-site stimulation. CONCLUSION: Frontoparietal rds-ccPAS may be a better treatment option for GAD.

User-Guided Deep Human Image Matting Using Arbitrary Trimaps.

Image matting is widely studied for accurate foreground extraction. Most algorithms, including deep-learning based solutions, require a carefully edited trimap. Recent works attempt to combine the segmentation stage and matting stage in one CNN model, but errors occurring at the segmentation stage lead to unsatisfactory matte. We propose a user-guided approach for practical human matting. More precisely, we provide a good automatic initial matting and a natural way of interaction that reduces the workload of drawing trimaps and allows users to guide the matting in ambiguous situation. We also combine the segmentation and matting stage in an end-to-end CNN architecture and introduce a residual-learning module to support convenient stroke-based interaction. The proposed model learns to propagate the input trimap and modify the deep image features, which can efficiently correct the segmentation errors. Our model supports arbitrary forms of trimaps from carefully edited to totally unknown maps. Our model also allows users to choose from different foreground estimations according to their preference. We collected a large human matting dataset consisting of 12K real-world human images with complex background and human-object relations. The proposed model is trained on the new dataset with a novel trimap generation strategy that enables the model to tackle different test situations and highly improves the interaction efficiency. Our method outperforms other state-of-the-art automatic methods and achieve competitive accuracy when high-quality trimaps are provided. Experiments indicate that our interactive matting strategy is superior to separately estimating the trimap and alpha matte using two models.

Context-Consistent Generation of Indoor Virtual Environments Based on Geometry Constraints.

In this article, we propose a system that can automatically generate immersive and interactive virtual reality (VR) scenes by taking real-world geometric constraints into account. Our system can not only help users avoid real-world obstacles in virtual reality experiences, but also provide context-consistent contents to preserve their sense of presence. To do so, our system first identifies the positions and bounding boxes of scene objects as well as a set of interactive planes from 3D scans. Then context-consistent virtual objects that have similar geometric properties to the real ones can be automatically selected and placed into the virtual scene, based on learned object association relations and layout patterns from large amounts of indoor scene configurations. We regard virtual object replacement as a combinatorial optimization problem, considering both geometric and contextual consistency constraints. Quantitative and qualitative results show that our system can generate plausible interactive virtual scenes that highly resemble real environments, and have the ability to keep the sense of presence for users in their VR experiences.

Simulating Fractures With Bonded Discrete Element Method.

Along with motion and deformation, fracture is a fundamental behaviour for solid materials, playing a critical role in physically-based animation. Many simulation methods including both continuum and discrete approaches have been used by the graphics community to animate fractures for various materials. However, compared with motion and deformation, fracture remains a challenging task for simulation, because the material's geometry, topology and mechanical states all undergo continuous (and sometimes chaotic) changes as fragmentation develops. Recognizing the discontinuous nature of fragmentation, we propose a discrete approach, namely the Bonded Discrete Element Method (BDEM), for fracture simulation. The research of BDEM in engineering has been growing rapidly in recent years, while its potential in graphics has not been explored. We also introduce several novel changes to BDEM to make it more suitable for animation design. Compared with other fracture simulation methods, the BDEM has some attractive benefits, e.g., efficient handling of multiple fractures, simple formulation and implementation, and good scaling consistency. But it also has some critical weaknesses, e.g., high computational cost, which demand further research. A number of examples are presented to demonstrate the pros and cons, which are then highlighted in the conclusion and discussion.

Hierarchical Generation of Human Pose With Part-Based Layer Representation.

Human pose transfer has been becoming one of the emerging research topics in recent years. However, state-of-the-art results are still far from satisfactory. One main reason is that these end-to-end methods are often blindly trained without the semantic understanding of its content. In this paper, we propose a novel method for human pose transfer with consideration of the semantic part-based representation of a human. In particular, we propose to segment the human body into multiple parts, and each of them represents a semantic region of a human. With the proposed part-based layer generators, a high-quality result is guaranteed for each local semantic region. We design a three-stage hierarchical framework to fuse local representations into the final result in a coarse-to-fine manner, which provides adaptive attention for global consistency and local details, respectively. Via exploiting spatial guidance from 3D human model through the framework, our method can naturally handle the ambiguity of self-occlusions which always causes artifacts in previous methods. With semantic-aware and spatial-aware representations, our method outperforms previous approaches quantitatively and qualitatively in better handling self-occlusions, fine detail preservation/synthesis and a higher resolution result.